Generalized symmetric mutual information applied for the channel capacity 
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' The channel capacity for the binary symmetric channel is investigated based on the symmetrized 

, definition of the mutual information, which is arising from an attempt of extension of information 

. content based on the nonadditivity. The negative capacity can emerge as an avoidable consequence 

^ ' for the generalization of the concept of the information entropy when g > 1. 
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0^ \ There has been a wide range of applications of the generalized (nonextensive) entropy by Tsallis |^ to physical 
systems which exhibit power-low behaviour. Recently, Landsberg et.al. |^ have focused on a possible constraint 
on the generalization of entropy concept and have compared channel capacities based on Shannon, Renyi, Tsallis, 
, and another generalized entropy by applying to a binary symmetric channel (BSC) for a fixed error probability e 
O of communication. It would be worth noting that the last generalized entropy introduced by Landsberg et al. , 
^ \ whose form takes as the Tsallis one divided by a factor ^ • (we refer to it as the modified form of Tsallis entropy 
^ ■ hereafter); the pi is the probability of the i-th state of the system considered and q d TZ, was presented independently 
by Rajagopal et al. from a consideration of form invariant structure of nonadditive entropy. 

Motivated by these status, the present author has attempted to develop the source coding theorem based on a 
nonadditive information content |4| from the fact that the information entropy has the same form as the statistical 
, mechanical entropy except for the Boltzmann factor in conventional definition. There we have chosen the nonadditive 
' information content as — \nqp{x) where In^ j; = (x^^'^ — !)/(! ~ ?)■ One should also notice that the normalized 
0'— average |^ of — \nqp{x) (information entropy) gives the modified form of Tsallis entropy |Q 



^p'^{x)lnqp{x) l-^p'^{x) 



O ' u (X) — '^^"^ — 

xeH xeH 

^ where Ti denotes alphabet to which a symbol x belongs. The concept of the mutual information can be regarded as 
CNj the important ingredient to evaluate a communication channel in information theory The mutual information 

is a measure of the amount of information that one random variable contains about another random variable. In 
both previous considerations (^,^, there is a point that should be investigated about the definition of the mutual 
information. In ||p they devised to take the average of conditional entropy H{Y \ X = x) using usual weighting p{x) 
I from the requisition that the capacity based on the any information entropy should be for e — 1/2 as Shannon's 
case. However this selection possesses asymmetry of the mutual information in X and Y unless we choose the Shan- 
"Y"; • non's H(Y \ X — x). The same applies to the case of |^], where we followed the usual definition of the reduction in 
■ uncertainty due to another variable(the entropy subtracted by the conditional entropy). In the nonadditive context, 
we may allow the mutual information to be not symmetric, however, if we decide to take the standpoint that any gen- 
eralization of mutual information has to retain symmetry, we would have to accept the modification of the intuitively 
' O ' reasonable property of Shannon's capacity. 
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In this Letter we propose a new definition of the mutual information which holds symmetry in X and Y in the line 
of the previous work and investigate the channel capacity of the BSC. 
^ Now we consider the mutual information indexed by q as 

"3 : Y) Hq{X) -[l + {q- l)Hq{Y)]Hq{X I Y) (2) 

, 

where Hq{X | y) is a nonadditive conditional entropy and is related with the nonadditive joint entropyiJg(X, y) as 
Hq{X,Y) = HqiY) + Hq{X \ Y) + {q - l)Hq{Y)Hq{X I Y) djU. Then wc immediately see that the mutual information 
becomes symmetric in X and Y, 

Iq{X-Y) = Iq{Y;X) ^ Hq{X) + Hq{Y) - Hq{X,Y). (3) 

In BSC TL — {0, 1} and the output (y) of the input code {x), and 1 can be received as a 1 and a respectively 
with the error probability e due to the external noise H. That is the conditional probabilities are given as p(0 | 1) = 



1 



p(l I 0) = e , p(0 I 0) = p(l I 1) = 1 — e respectively. We write the input (output) probability which we observe a 
and a 1 as po and pi {p'q and p'l) respectively. Then we have the joint probabilities Pxy = p{y \ x)px] Poo = (1 ~ ^)Po, 
Poi = epo, Pio = epi, and pn = (1 - 
Using this the nonadditive joint entropy y, 

is calculated as (1 - +p?))(e9 + (1 - e)9))/(g - 1). 

We define a generalized channel capacity as follows 

=^max/q(X;r). (5) 

p(x) 

Since the output probabilities pg and p[ are given as 

Po =Poo +Pio =Po + (pi -Po)e, p[ ^ Poi + Pn = Pi + {Po - Pi)e (6) 

respectively, the summation J^iPiVi) is calculated as ((1 — 2e)po + e)"^ + ((2e — l)po + 1 — e)"^. Then from Eq(||) the 
mutual information is expressed as 

9-1 (Po + (1 --Po)') V e9 + (l-e)V /(po,e) 

where we have put /(po, e) = ((1 — 2e)po + e)"^ + ((2e — l)po + 1 — e)"^. The Iq{X; Y) can be taken its maximum when 
Po = Pi = 1/2 for some ranges of q and e. Then the capacity is written as 

1 / 2«"i \ 

C„ = 2^-1 . (8) 

^ q-l\ e9 + (l-e)9y ' 

Fig.l shows the channel capacity Cq against e for the case of Shannon and different values of q. The capacity does 
not become zero for e = 1/2 except for Shannon. It exceeds the Shannon's capacity for the intermediate noise level 
{q < 1) and is below Shannon when the channel is very noisy(or less noisy). We note that the capacity can be negative 
also for our new definition of the mutual information for some ranges of q and e and we can have the capacity above 
Shannon for extreme cases of e. However, the negative channel capacity has a difficulty to understand in the usual 
communication framework as pointed out in the definition of [^. 

Summarizing, we have presented a new possible definition of the mutual information in nonadditive context, which 
preserves the symmetry in the input and the output. As an application of this, the behaviour of the channel capacity 
of BSC has shown for different noise characteristics. The generalized channel capacity can entail negativity for q > I 
when we sacrifice the symmetry of mutual information maintaining the zero capacity for e = 1/2 and vice versa. The 
necessities for investigating whether or not the capacity based on the statistics breaking the additivity can be valid 
for real communication channel remains unchanged. 
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Figure Caption 



FIG. 1. The channel capacity of BSC is plotted against the error probability for Shannon and some different values of q. 
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